The present invention relates to a new and improved construction of an apparatus for, and a method of, automatically and passively focusing a video camera with respect to a predeterminate object.
The present invention thus generally relates to the fields of optics and electronics and, specifically, to the field of image processing techniques. In its more particular aspects, the present invention specifically relates to a new and improved construction of an apparatus for, and a method of, automatically and passively focusing a video camera with respect to a predeterminate object and for producing an optimumly focused image of the predeterminate object in order to use such video image for controlling the operation of a manipulator unit. In this manner the video camera can be used for monitoring the operation of an industrial robot which constitutes the manipulator unit or an essential part thereof. The video camera may also be used for controlling the operation of a manipulator unit which constitutes or comprises an automatic target tracking unit or device and thus serves or cooperates with a so-called video-tracker contained therein. In both areas or fields of use the video camera may constitute a FLIR camera, i.e. a forward looking infrared camera which responds to heat radiation received thereby.
Means for automatically focusing the image in connection with a camera are well-known. Generally, a differentiation is made between active and passive focusing means and methods.
A first known method relies upon a signal, e.g. an ultrasonic or infrared beam of rays which is emitted by the camera. The ultrasonic or infrared radiation reflected by the object to be imaged is received and generates a signal which is utilized for focusing the camera with respect to such object. When using ultrasonic radiation, the time difference between the emission of the signal and the reception of the echo signal reflected by the object can be used for determining the distance of the object from the camera. When relying upon infrared radiation, the reflected signal is received by means of a photodiode which is positioned laterally of the transmitter at the camera and the distance of the object is determined from a parallax angle. The focus adjustment is then carried out on the basis of the functional relationship between the distance of the object and the distance of the associated image.
Such known methods thus rely upon an active distance measurement which is well suited for the non-professional field and can be realized using apparatus of relatively limited complexity. For industrial uses, however, problems may occur due to the presence of interfering sources which cause an interference or disturbance in the reflected radiation originating from the object. Furthermore, for military purposes, any active target measurement or tracking is out of question a priori for camouflage reasons.
A second known method derives data concerning the focusing condition directly from the received image on the basis of a predetermined criterion. The adjustment of the lens system is corrected until an optimum adjustment is achieved which corresponds to the predetermined focusing criterion.
From the point of view of optical imaging theory it would be obvious to base the focusing criterion upon the so-called space-frequency analysis or spatial filtering. Such analysis relies upon a distinct property of convex lenses, namely the property of transforming into the Fourier transform an image which is illuminated by monochromatic and parallel as well as temporally and spatially coherent radiation. This Fourier transform, the so-called space-frequency image is positioned in the focal plane of the imaging lens and corresponds to a frequency analysis of the illuminated image structure. Coarse structures of the image, i.e. coarse structures in the imaged object result in low space or spatial frequencies and fine structures of the image, i.e. fine structures in the imaged object, result in high space or spatial frequencies. The latter high space or spatial frequencies are only present in a well-focused image since such fine structures are only reproduced in a sharp image. The appearance of high space or spatial frequencies thus is a useful criterion for optimum focusing of the lens or lens system. In practice, however, the use of such method is fraught with difficulties since, on the one hand, white light, i.e. non-monochromatic and non-coherent radiation is nearly always used for illuminating the object so that there occurs an intermixing of the higher space or spatial frequencies and, on the other hand, the measurement of the light intensity distribution in the focal plane must be carried out at very high precision.
Frequently, therefore, the focusing of photo cameras is carried out on the basis of other criteria. For example, the image section or frame which is to be focused, can be imaged through two lenses onto an image sensor comprising a chain or sequence of charge-coupled devices. The position of the focus with respect to the image plane is determined from the distance between the two image structures corresponding to the two imaging lenses. More detailed information on such systems are available from the data sheets of related photo cameras.
When using cameras employing electronic image recording such as, for example, video cameras, the already mentioned image analyses by means of charge-coupled devices are very frequently utilized. The reason therefor is that the other aforementioned active methods which are based on ultrasonic or infrared radiation, are afflicted with insufficient reliability. The interfering effects produced by glass plates and inclined surfaces at the object to be imaged, upon reflected infrared radiation are here specifically mentioned.
Automatic focusing means in connection with cameras for non-industrial uses are primarily to be considered as facilitating the operation of such cameras in terms of increasing the operator comfort but otherwise there can hardly be attributed thereto any positive significance. The situation, however, is different with respect to cameras which are utilized in the industrial field, for example, in connection with robot equipment or in the event of use for military purposes. In such cases the operator simply cannot be expected to be charged with the task of focusing. One reason therefore is that other tasks have priority and another reason can be that, in the event of rapidly moving objects, the operator does not have enough time at his disposal for readjusting the focusing. Therefore, efforts for automating the focusing operation are more than merely desirable. Still, hitherto no means or methods have become known for solving this problem in a fully satisfying manner.
An important requirement for any focusing means or method is the recognition of the object with respect to which the focusing operation must be carried out relative to the background which is tolerated to be imaged out of focus. This problem is without significance with respect to non-industrial or non-military uses because in those fields the image area to be focused is determined by the operator by means of the view finder.
It is quite conceivable that, in the video camera image of an industrial robot, the movable object like, for example, a workpiece or a tool operating upon a workpiece, can be hardly recognized or not at all relative to the background for certain periods of time. This problem is particularly encountered in the case of military targets which move across a continuously changing background. In such cases the object recognition has great significance. The object recognition, however, will not be discussed in detail herein because the object recognition requires optimization of brightness and contrast in the video image and this is the subject of the initially mentioned and cross-referenced, commonly assigned and copending United States application.